SherbyNet - /page/wikibook/entry.mu

Micron Document

<!DOCTYPE html>

<title>Many-task computing</title>

</head>

Many-task computing

</h1>

</header>

/* start https://en.wikipedia.org/ */

.mw-parser-output .ambox{border:1px solid #a2a9b1;border-left:10px solid #36c;background-color:#fbfbfb;box-sizing:border-box}.mw-parser-output .ambox+link+.ambox,.mw-parser-output .ambox+link+style+.ambox,.mw-parser-output .ambox+link+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+style+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+link+.ambox{margin-top:-1px}html body.mediawiki .mw-parser-output .ambox.mbox-small-left{margin:4px 1em 4px 0;overflow:hidden;width:238px;border-collapse:collapse;font-size:88%;line-height:1.25em}.mw-parser-output .ambox-speedy{border-left:10px solid #b32424;background-color:#fee7e6}.mw-parser-output .ambox-delete{border-left:10px solid #b32424}.mw-parser-output .ambox-content{border-left:10px solid #f28500}.mw-parser-output .ambox-style{border-left:10px solid #fc3}.mw-parser-output .ambox-move{border-left:10px solid #9932cc}.mw-parser-output .ambox-protection{border-left:10px solid #a2a9b1}.mw-parser-output .ambox .mbox-text{border:none;padding:0.25em 0.5em;width:100%}.mw-parser-output .ambox .mbox-image{border:none;padding:2px 0 2px 0.5em;text-align:center}.mw-parser-output .ambox .mbox-imageright{border:none;padding:2px 0.5em 2px 0;text-align:center}.mw-parser-output .ambox .mbox-empty-cell{border:none;padding:0;width:1px}.mw-parser-output .ambox .mbox-image-div{width:52px}@media(min-width:720px){.mw-parser-output .ambox{margin:0 10%}}@media print{body.ns-0 .mw-parser-output .ambox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style>

Many-task computing (MTC)<a href="#cite_note-1">[1]</a><a href="#cite_note-2">[2]</a><a href="#cite_note-3">[3]</a><a href="#cite_note-4">[4]</a><a href="#cite_note-5">[5]</a><a href="#cite_note-:0-6">[6]</a><a href="#cite_note-7">[7]</a> in <a href="Computational_science" title="Computational science">computational science</a> is an approach to <a href="Parallel_computing" title="Parallel computing">parallel computing</a> that aims to bridge the gap between two <a href="Computing_paradigm" class="mw-redirect" title="Computing paradigm">computing paradigms</a>: <a href="High-throughput_computing" title="High-throughput computing">high-throughput computing</a> (HTC)<a href="#cite_note-8">[8]</a> and <a href="High-performance_computing" title="High-performance computing">high-performance computing</a> (HPC).

<div class="mw-heading mw-heading2"><h2 id="Definition">Definition</h2></div>

MTC is reminiscent of HTC, but it "differs in the emphasis of using many computing resources over short periods of time to accomplish many computational tasks (i.e. including both dependent and independent tasks), where the primary metrics are measured in seconds (e.g. FLOPS, tasks/s, MB/s I/O rates), as opposed to operations (e.g. jobs) per month. MTC denotes high-performance computations comprising multiple distinct activities, coupled via file system operations. Tasks may be small or large, uniprocessor or multiprocessor, compute-intensive or <a href="Data-intensive_computing" title="Data-intensive computing">data-intensive</a>. The set of tasks may be static or dynamic, homogeneous or heterogeneous, loosely coupled or tightly coupled. The aggregate number of tasks, quantity of computing, and volumes of data may be extremely large. MTC includes loosely coupled applications that are generally communication-intensive but not naturally expressed using standard message passing interface commonly found in HPC, drawing attention to the many computations that are heterogeneous but not "happily" parallel".<a href="#cite_note-:0-6">[6]</a>

Raicu et al. further state: "There is more to HPC than tightly coupled MPI, and more to HTC than <a href="Embarrassingly_parallel" title="Embarrassingly parallel">embarrassingly parallel</a> long running jobs. Like HPC applications, and science itself, applications are becoming increasingly complex opening new doors for many opportunities to apply HPC in new ways if we broaden our perspective. Some applications have just so many simple tasks that managing them is hard. Applications that operate on or produce large amounts of data need sophisticated data management in order to scale. There exist applications that involve many tasks, each composed of tightly coupled MPI tasks. Loosely coupled applications often have dependencies among tasks, and typically use files for inter-process communication. Efficient support for these sorts of applications on existing large scale systems will involve substantial technical challenges and will have big impact on science."<a href="#cite_note-:0-6">[6]</a>

<div class="mw-heading mw-heading2"><h2 id="Related_areas">Related areas</h2></div>

Some related areas are multiple program multiple data (MPMD), high throughput computing (HTC), workflows, capacity computing, or <a href="Embarrassingly_parallel" title="Embarrassingly parallel">embarrassingly parallel</a>. Some projects that could support MTC workloads are Condor,<a href="#cite_note-9">[9]</a> <a href="Mapreduce" class="mw-redirect" title="Mapreduce">Mapreduce</a>,<a href="#cite_note-10">[10]</a> Hadoop,<a href="#cite_note-11">[11]</a> <a href="Boinc" class="mw-redirect" title="Boinc">Boinc</a>,<a href="#cite_note-12">[12]</a> Cobalt<a href="#cite_note-13">[13]</a> HTC-mode,<a href="#cite_note-14">[14]</a> Falkon,<a href="#cite_note-15">[15]</a> and Swift.<a href="#cite_note-16">[16]</a><a href="#cite_note-17">[17]</a>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist">

<li id="cite_note-1"><a href="#cite_ref-1">^</a> IEEE Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS08) 2008, <a rel="nofollow" class="external free" href="http://datasys.cs.iit.edu/events/MTAGS08/">http://datasys.cs.iit.edu/events/MTAGS08/</a>

</li>

<li id="cite_note-2"><a href="#cite_ref-2">^</a> ACM Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS09) 2009, <a rel="nofollow" class="external free" href="http://datasys.cs.iit.edu/events/MTAGS09/">http://datasys.cs.iit.edu/events/MTAGS09/</a>

</li>

<li id="cite_note-3"><a href="#cite_ref-3">^</a> IEEE Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS10) 2010, <a rel="nofollow" class="external free" href="http://datasys.cs.iit.edu/events/MTAGS10/">http://datasys.cs.iit.edu/events/MTAGS10/</a>

</li>

<li id="cite_note-4"><a href="#cite_ref-4">^</a> ACM Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS11) 2011, <a rel="nofollow" class="external free" href="http://datasys.cs.iit.edu/events/MTAGS11/">http://datasys.cs.iit.edu/events/MTAGS11/</a>

</li>

<li id="cite_note-5"><a href="#cite_ref-5">^</a> IEEE Transactions on Parallel and Distributed Systems, Special Issue on Many-Task Computing, June 2011, <a rel="nofollow" class="external free" href="http://datasys.cs.iit.edu/events/TPDS_MTC/">http://datasys.cs.iit.edu/events/TPDS_MTC/</a>

</li>

<li id="cite_note-:0-6">^ <a href="#cite_ref-:0_6-0">a</a> <a href="#cite_ref-:0_6-1">b</a> <a href="#cite_ref-:0_6-2">c</a> I. Raicu, I. Foster, Y. Zhao. "Many-Task Computing for Grids and Supercomputers", IEEE Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS08), 2008

</li>

<li id="cite_note-7"><a href="#cite_ref-7">^</a> "Many Task Computing: Bridging the performance-throughput gap", International Science Grid This Week (iSGTW), January 28th, 2009, <a rel="nofollow" class="external free" href="http://www.isgtw.org/?pid=1001602">http://www.isgtw.org/?pid=1001602</a> <a rel="nofollow" class="external text" href="https://web.archive.org/web/20110101121519/http://www.isgtw.org/?pid=1001602">Archived</a> 2011-01-01 at the <a href="Wayback_Machine" title="Wayback Machine">Wayback Machine</a>

</li>

<li id="cite_note-8"><a href="#cite_ref-8">^</a> M. Livny, J. Basney, R. Raman, T. Tannenbaum. "Mechanisms for High Throughput Computing," SPEEDUP Journal 1(1), 1997

</li>

<li id="cite_note-9"><a href="#cite_ref-9">^</a> D. Thain, T. Tannenbaum, M. Livny, "Distributed Computing in Practice: The Condor Experience" Concurrency and Computation: Practice and Experience 17( 2–4), pp. 323–356, 2005

</li>

<li id="cite_note-10"><a href="#cite_ref-10">^</a> J. Dean, S. Ghemawat. "MapReduce: Simplified data processing on large clusters." In OSDI, 2004

</li>

A. Bialecki, M. Cafarella, D. Cutting, O. O'Malley. "Hadoop: A Framework for Running Applications on Large Clusters Built of Commodity Hardware," <a rel="nofollow" class="external free" href="https://lucene.apache.org/hadoop/">http://lucene.apache.org/hadoop/</a> <a rel="nofollow" class="external text" href="https://web.archive.org/web/20070210202041/http://lucene.apache.org/hadoop/">Archived</a> 2007-02-10 at the <a href="Wayback_Machine" title="Wayback Machine">Wayback Machine</a>, 2005

</li>

<li id="cite_note-12"><a href="#cite_ref-12">^</a> D.P. Anderson, "BOINC: A System for Public-Resource Computing and Storage," IEEE/ACM International Workshop on Grid Computing, 2004

</li>

<li id="cite_note-13"><a href="#cite_ref-13">^</a> <a rel="nofollow" class="external free" href="http://wiki.bgl.mcs.anl.gov/wiki/ResourceMgrs/Cobalt">http://wiki.bgl.mcs.anl.gov/wiki/ResourceMgrs/Cobalt</a>

</li>

<li id="cite_note-14"><a href="#cite_ref-14">^</a> IBM Corporation. "High-Throughput Computing (HTC) Paradigm," IBM System Blue Gene Solution: Blue Gene/P Application Development, IBM RedBooks, 2008

</li>

<li id="cite_note-15"><a href="#cite_ref-15">^</a> I. Raicu, Y. Zhao, C. Dumitrescu, I. Foster, M. Wilde. "Falkon: A Fast and Lightweight Task Execution Framework," IEEE/ACM SC, 2007

</li>

<li id="cite_note-16"><a href="#cite_ref-16">^</a> Y. Zhao, M. Hategan, B. Clifford, I. Foster, G. Laszewski, I. Raicu, T. Stef-Praun, M. Wilde. "Swift: Fast, Reliable, Loosely Coupled Parallel Computation", IEEE SWF, 2007

</li>

<li id="cite_note-17"><a href="#cite_ref-17">^</a> M. Wilde, M. Hategan, J. M. Wozniak, B. Clifford, D. S. Katz, and I. Foster." Swift: A language for distributed parallel scripting." Parallel Computing, 37:633–652, 2011.

</li>

</ol></div></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-06-19" href="https://en.wikipedia.org/wiki/?title=Many-task_computing&oldid=1296417541">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>